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C\j© Method of loading surface mounted device and an apparatus therefor. 

°® A method m(i an apparatus for cresting surface mounted device such as the IC (31) or the like by the hand 
EL (33) of the robot and automatically and accurately loading them onto the contact pattern of the printed boanJ by 
means of the image processor (7) are disclosed. According to this method, the measurement line (17) is 
provided onto each image pattern of the pin array of the surface mounted device and the array of the contact 
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the pin array and the array of the contact.pattem is sought from the variation of the brightness (l.ll,...) on that 
line. In accordance with the result thus obtained, displacement of the position and that of the rotation over the 
entire surfaced part and the contact pattern are sought and the operation of the robot (1-10) is controlled in 
accordance wtth-that information. 

FIG. 1 
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METHOD OF LOADING SURFACE MOUNTED DEVICE AND AN APPARATUS THEREFOR 



BACKGROUND OF THE INVENTION: 

FIELD OF THE INVENTION " 

6 

The present invention relates to a method of automatic loading of surface mounted device such as the 
flat package type i..i;egrated circuit (IC) to the printed board and an apparatus therefor, and particularly to a 
method of automatic loading of the surface mounted device and an apparatus therefor, in which the 
automatic loading is conducted by visually positioning them. 

10 

DESCRIPTION OF THE PRIOR ART 

Conventionally, with reference to the method and the apparatus for mounting the electronic parts to 
75 printed circuit board, there are examples described in Japanese Patent Laid-Open No. 1900/1985. 

According to this embodiment, a special mark is previously provided on the board for positioning 
electronic parts onto the board and this mark is recognized for* positioning parts on the board- 
In the above-mentioned prior art, the special mark other than the contact pattern is put onto the printed 
circuit board and the mark is read and recog nized, which forms a problem. Furthermore, the TV camera is 
20 required to moved in a special way to pick up the imaged that special mark. Namely, the pickup field of 
the TV camera in principle covers the entire printed circuit board, while the special mark must be provided 
in a smail size on a part of the board and for the detection of this smail mark, the prior art has picked up 
the zoomed image of several mark by moving the TV camera, which therefore, has caused the alteration of 
the design of the board and the delay of the recognizing speed due to moving the TV camera. 
25 ' 

SUMMARY OF THE INVENTION: 

An object of .the present invention is to provide a method of loading :^e surface mounted devices and 
oo an apparatus therefor, which allows them to be automatical loaded accurately as well as promptly. 

Another object of the present invention is to provide a method of loading the surface mounted devices 
orrto the board and a method therefor, wherein, even in case the pattern of leads (pins) or contact patterns 
are irregular, the central position of the edge points not symmetric of leads or contacts can be accurately 
caiculated, so that the positioning free of any error may be possible. 
35 A further object of the present invention is to provide a method of loading and an apparatus therefor, in 
which bent leads can be detected at a high speed accurately and easily. 

For this purpose, in view of the fact that, in case the absolute position and orientation of the contact 
pattern of the surface mounted devices and the contacts on the printed circuit board onto which surface 
mounted devices boards are loaded can be accu lately obtained respectively, relative position and orienta- 
te tion among them are also, determined and surface mounted perts can be accurately loaded, the present 
invention aims at visually obtaining the absolute position and orientation of the surfaced parts and the 
contact pattern from the pin pattern and the contact pattern which can be generally called the contact 
pattern. 

Furthermore, data on the portions where a straight line on the arrangement of the leads cross them is 
45 extracted from -*ata on the planar image of the IC and the bendings of leads are detected from the pattern 
of the gray scale brightness data thus extracted and further, the irregular pattern or even a partial absence 
of the array of the pattern of leads or contact can be corrected. 



so BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a view, illustrating an embodiment of the present invention. Fig. 2 show an external view of 
the loading machine. Fig. 3 shows a device loading processing procedure. Fig. 4 shows a flow chart of the 
device loading. Fig. 5 shows a top view of the printed board. Rg. 6 shows a top view of the integrated 
circuit and the flat packaged IC. Rg. 7 shows on explanatory view of how the inclination is calculated by the 
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measurement line in the pattern of contacts. Figs. 8 to 10 show the pattern of brightness along the 
measurement line. Fig. 11 shows an exemplary view of the actual contact pattern. Fig. 12 and 13 
respectively shows the relationship between the edge coordinate of the contact pattern and the brightness 
pattern on the measurement line. Fig. 14 shows a view, illustrating the succesdingly edge points within the 
gradient of the concentration taken in the same direction. Fig. 15 shows a view, viewed from the three 
directions of one example of the flatly package type IC in which the omitted modification Is needed. Fig. 16 
• shows an explanatory view, of the irregular leads array in which .the omitted modification is needed." Fig. 17 
shows a view, illustrating the amount of correction Ap in the method of measureing the omitted correction. 
Fig. 18 shows a view, illustrating the method of measuring the position and the orientation of the entire 
jo conrtact pattern having the example of contacts in the four and two directions. Fig. 19 shows a view, 
illustrating the difference of the waveform of the brightness according to the direction'of the measurement 
line, the scanning direction of the screen and the direction vertical thereto Fig. 20 shows a flow chart, 
illustrating the processing teaching procedure in one embodiment of the present invention. Fig. 21 shows a 
flow chart, illustrating the procedure measurement time of measurement in one embodiment of the present 
invention. Fig. 22 shows an explanatory view, illustrating the calculation of the contact pattern and its central 
position. Fig, 23 shows a plan view, illustrating one example of the IC leads which are bent Fig. 24 shows 
an explanatory view, illustrating the pickup operation in one embodiment of the method of detecting the 
bent leads of the IC according to the present invention. Fig. 25 shows an explanatory view, illustrating the 
detecting principle, in one example, of the method of detecting bent leads of the IC according to the 
present invention. Fig. 2S shows an explanatory view, illustrating the optical system at the time of image 
acquisition In one embodiment Fig. 27 also shows a flow chart for explaining the operation in one 
embodiment ~ 
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25 DESCRIPTION OF THE PREFERRED EMBODIMENTS: 

The invention is hereinafter described in more detail with respect to the drawings, and prior to that, the 
theoretical background is briefly described. 

In general, since the array of the pixels of the image is discrete, the minimum unit of the position to be 
30 detected from the image is a distance between pixies. However, in case of obtaining the position or 
orientation of a certain object It is possible to detect the position or the orientation with a higher accuracy 
by averaging a plurality of sample points on the same object The more the number of individual sampling 
points is, the accuracy becomes ;,igher, while,, on the contrary, the processing speed cannot but be 
decreased. 

as By the way. in the surface mounted devices or in the contact pattern on which they are loaded.' the 
width of pins and contacts is narrow, but their number is large, and further, their width is set constant and 
they are arrayed on the straight line. Furthermore, their approximate position and the orientation are 
previosuly fixed by the mechanical positioning. Consequently, it is possible to draw a sampling straight line 
within the screen which will intersect at right angles with the pin array or the contact array of the surfaced 

40 parts. The brightness of the image on this straight line draws a waveform in correspondence to the array of 
pins or contacts and the position of the surface mounted devices or the contact pattern can be obtained In 
the direction of the sampling straight fine. In general, since there are two or more arrays of pins or contacts, 
their whole position and the orientation can be accurately obtained by the combination of two or more 
arrays. ' 

45 The present invention is now described more specifically. 

Fig. 1 shows a construction of the apparatus for automatically loading the IC onto the circuit board - 
(surface mounter). This automatic loading apparatus consists two television cameras 1 and 2, a cartesian 
coordinate type robot 3 for mounting the IC, a board supply apparatus 4, an IC supply apparatus 5, a rough 
positioning apparatus 6, an image processor 7, a robot control device & a sequencer 9 and an operatino 

so unit 10. » 

The sequencer 9 forms a center of the entire control and performs the sequence control in accordance 
with the predetermined sequence program. The instruction of this sequencer controls the robot control 
device 8, the board supply apparatus 4 and the rough positioning apparatus 6. Since, for te operation, the 
sequencer 9 needs the signal sent from the robot control device 8, it takes in the signal from the robot 
S3 control unit 8. " 
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The robot control devicve 8 performs the control of the cartesian coordinate type robot 3 for mounting 
the IC and the control of the IC supply apparatus 5. The image processer 7 acquires the image picked up 
from the television camera 1 and 2 and processes the patterns of the printed board and the IC, so that the 
information for positioning the printed board and IC and loading them may be obtained. Such results are fed 
5 to the robot control device, so that the cartesian coordinate type robot 3 for mounting the IC and the IC 
supply apparatus 5 may be controlled, and also the board supply apparatus 4 may be controlled through 
the sequencer 9. - - - 

The television camera 1 is a camera for picking up the images of the vacuum gripper for grasping the 
IC or the board and the television camera- 2 for detecting the position of the !C. The cameras 1 and 2 
w correspond to the cameras B and E respectively, illustrated in Fig. 3 to be described later. 

The operating unit 10 is for the man-machine system, comprising a keyboard and a CRT and performs 
man-machine operation the image processor 7 and the robot control device 8. The board supply apparatus 
4 supplys the printed board to the loading position. The IC supply apparatus 5 suppiys the IC from the IC 
loading position to the grasping position of the robot hands. The rough positioning apparatus 6 decides the 
75 rough position of the IC. 

Fig. 2 shows an externa] view of the loading apparatus. This loading apparatus comprises the cartesian 
coordinate type robot 3, board supply apparatus 4, IC supply apparatus 5, image processor 7. robot control 
device 8 and the sequencer 9, which are set on a common base. The television cameras 1 and 2 and the 
operating unit 10 are omitted. 
20 The robot may be also of an articulated or skalar type rf it has a sufficient positioning accuracy, and to 
its hand are mounted the vacuum gripper for grasping the IC as the surface mounted device and the TV 
camera for picking up the board image. The other TV camera for detecting the position of the IC is fixedly 
mounted on the base in such a way that it may face upwardly. 

An outline of the loading apparatus is as follows and its fundamental operation is described in Rg. 3. 
25 furthermore, the loading procedure is shown in Rg. 4, 

(a) Rrst. the board A is mechanically fixded (100) to the robot hand C on the board supply apparatus 
4. The TV camera B is moved onto the board (102) to obtain the accurate position of the contact pattern for 
loading the IC on the board A (104). 

(b) The hand C is moved to the IC rough! positioning apparatus (106) and chucks the IC (integrated 
3c circuit) D from this apparatus, while in case the hand has the rough positioning function, it chucks the IC 

directly from the IC supply apparatus (I08). 

<c) The IC D is moved onto the TV camera E for the detection of the position of the IC (HO), so that 
the divergence between the IC D and the hand C may be detected (112). 

(d) The position of the IC and the contact pattern which is obtained by the image processor 7 is fed 
5S back via the robot control device 8 to the robot, so that the IC may be aiignedly loaded onto the contact 
pattern on the beard (114). 

. As described above, positioning is conducted by the image processor. How that processing is made is 
now hereinafter described. 

When the contact pattern on the printed board and the array of feet of the leads on the IC fiat package. 
40 which are to be detected, are picked up by the TV camera by the reflected light, they represent the bright 
portions, having a certain difference of the contrast from the background. The distribution of the brightness 
on the measurement line, which is set so that rt may intersect approximately vertically with respect to its 
pattern array, forms a waveform having a certain cycle corresponding to the ai,ay of the pattern. 

The amplitude of the brightness drawing the waveform is closely concerned with the frequency of the 
46 pattern array and the amplitude on the measurement line taken in the scanning direction of the television 
camera becomes smaller due to the condition of the measurement (for example, the array of pixels of the 
TV camera). However, if the contrast is greatly set and the threshold vaJue which can be regarded as the 
edge is suitably set beforehand, it can be easily seen where the pattern is located. 

The farther description is as follows. As the IC to be loaded, the surface mounting package (flat 
so package) IC is chosen, and the plan view and the etevationai view are respectively shown in Rg. 5, (a) and - 
(b) by way of example and the contact pattern on the board on which this IC 15 is loaded is shown in Rg. 6. 

As shown in the Rgs. f the IC 15 has four arrays of pins (leads) 15-1 to 15-4 and the contact pattern 16 
comprising pin arrays I6-I to 16-4 which correspond respectively to these contact patterns 15-1 to 15-4 is 
formed on the board. Though the high accuracy aJignmnet is necessary to accurately load the pin arrays of 
55 the IC on the array of the contact pattern in the corresponding state, the mechanical positioning by the 
board supply apparatus and the accuracy of the IC chucking by the robot are not enough. Consequently, 
the position is visually detected with a higher accuracy, so that the information on the detected position 
may be fed back via the robot control device to the robct 
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As described already, the detection of the position is conducted by means of the TV cameras and the 
image processor. One of TV cameras is mounted to the robot hand in such a way that it may face 
downwardly and the contact pattern on the board is picked up, while the other is fixed onto the base in such 
a way that it may face upwardly, so that the IC chucked by the hand positioned immediately thereabove; 
> The method of detecting the position from the image from the TV camera is described as follows with 
respect to Figs. 7 to 10. 

The array of the contact pattern and the array of the IC pins on the board are arranged on the image 
along the straight lines respectively in the longitudinal direction and in the direction formina right angles 
thereto. However, in the actual image, the wiring pattern and the background are further picked up together. 
) Now, if, as shown in Fig. 7 (a), (b). the measurement straight line 17 Is set along each array of the 
rectangular pattern in the direction in which it may intersect at right angles with the rectangle and the 
position of the measurement straight line 17 is fixed within the screen, the position of the pattern is 
somewhat offset with each board and IC, but it can be mechanically positioned to such an extent that it may 
necessarily intersect with the sampling straight line. In this case, the brightness of the image on the 
; measurement One 17. if the measurement line is assumed to be X-axis, as shown in Fig. 8, forms a 
waveform in conformity with the array of the pattern. Consequently, if the threshold reference threshold 
value B,„ between the pattern and the boundary is set and is compared with the above pattern of the 
waveform, the boundary position Xm, X w ,„.X,...on the measurement straight line is determined. As shown in 
Fig. 9, since, under the ideal condition, the waveform of the reflected light from the pattern forms a 
rectangle as shown as the brightness (I), the boundary of the pattern is the easiest to detect but in case the 
waveform is corrupt as in the brightness (ll). the crest of the mountain (maximum point) i is selected, so that 
the center of the pattern may be obtained. 

Now. a method of obtaining the boundary of the pattern is concretely described with respect to Fig. 10. 
When the lighting condition is favorable and the brightness of the screen is uniform . (brightness (III)), the 
threshold level of the brightness may be property set and the position of pixels at both ends of the brighter 
or darker range than the value thus obtained may be obtained as the boundary of the pattern. Furthermore, 
under the same lighting cndition brightness is substantially indentical with the briohtness (III) (brightness - 
(IV)), the gradient of the brightness between pixels at both sides of the threshold value is taken into 
consideration and the position showing the same brightness as the threshold value is obtained in terms of 
the real value, the accuracy is further improved. Furthermore, in case the illumination condition is not 
excellent and the brightness of the screen is not uniform (The brightness (V) represents that case, but in the 
case of other brightnesses (III) and (IV), too, the same can be conceived.) every pattern cannot be 
separated by a single threshold value, but in such a case, the position of the pixel at which the gradient of 
the bnghtness the between adjacent pixels becomes the greatest is obtained as the boundary, or the 
weighted mean is obtained for each extent from the mountain to the trough. 

The latter method (when the illumination condition is not excellent and the brightness of the screen is 
not uniform) is described as follows with respect to Figs. 20 and 21. 

Brightness (III) to (V) illustrated in Fig. 10 has various waveforms. Consequently, tine trough and the 
mountain can be defined in various ways. In this embodiment portions of the curve which are monoto- 
!l!!| S i.' nCr8aSin9 * decreasin 9 are noted and in the case cf the monotonous increment, its starting point is 
defined as the trough and its terminating point as the mountain. In the case of the monotonous decrement 
its starting point is defined as the mountain and its terminating point as the trough (See step 206 of fio 20 
and step 306 of Fig. 21). ■ 

Furthermore, in any of the monotonous increment and the monotonous decrement of the curve, it is 
necessary for the difference of the brightness between the crest and the trough to be above the permissible 
value D* (step 208, 308). This is because, if not viewed in that way, the crest and the trough, which do not 
show a very marked difference in the brightness between each other, are likely to be seen as the 
charactenznig pent of every pattern, often leading to the enoneous recognition. Therefore, the threshold 
value of the gradient of the concentration is to be previously selected (in other words, the edge threshold 
value of the bnghtness slope is to be previously determined) (steps 202, 302). 

Referring to the examples of brightness (III). (IV) and (V) in Fig. 10, the trough exists at the position of 

1 v. -S 9 CrSSt 31 th6 P ° Sifi0n a X '*'- N8Xtl me C00rdinat9 X* ai the portion of the edge through the 
weighted mean is obtained with respect to the brightness at all of the plurality of sampling points between 
those trough and th9 crest (steps 212, 312). 
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The above process can be expressed as follows. The expression whereby it is decided whether the 
difference is above the permissible value or not is as follows (steps 208, 308) 
f(N)-f(M)p>D th _ .(I) 

where: D th : threshold value of the gradient of the concentration (in other words, edge threshold value of 
brightness slope) 

M: starting point of the monotonous increment and decrement of the concentration of the brightness 
N: terminating point of the monotonous increment and the decremenj of the concentration of brightness * 
The expression by the weighted mean for detecting the edge is as follows (step 212, 312), 

N 

Z (X + ±)[|f(x. Al ) - f (X. ) J ] ' 
i=M 4 ~ ' x x 
= ^ : : . . . ( 2 ) 

I |f(x J - f (X j | 
i=M 1 x 1 

where: X*: coordinate value of the edge taken abng the measurement line 17 

f(x): brightness at the coordinate X 

f(x,):X 

20 In this case, X j means a sampling point existing between the sampling point M and the sampling point 

The coordinate of the edge by the weighted mean obtained by the expression (2) is obtained for 
each set of trough and crest (or crest and trough) (steps 214, 314). 

If there are n pair(s) of the troughs and the crests for a single pattern array, the average value U of the 
25 coordinate Xfc by the weighted mean of the n set(s) is calculated 

n 

I X, 



rs 
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k=l k 

U = — — (3) 

n 



This calculated position U is regarded as the central point o' cnc pattern an-ay. (steps 222 and 320). For 
other pattern arrays, the central points are also obtained, (steps 202 to 225, 302 to 322). 
M In order to perform The above-described calculation, the memory (not shown in the drawing) within the 
image processor is effectively utilized. First, the memory takes in and stores the image picked up from the 
TV camera. Next, a measurement line is set and the image data (data on the brightness) on the 
measurement line is read out to be stored temporarily within the work area (table) within the memory. This 
content stored is data on the wave-form as shown in Rg, 10. Namely, this table takes the position of the 
^ measurement line 17 (sampling position) as the address and the data relates to the data on the brightness at 
that position. Instead, the address may be independent of the position, while the data may be composed of 
the position and the brightness. 

Next, the content of this table is read out and the crest and the trough are found in accordance with the 
expression (I) arid the weighted mean is obtained for each pair of the crest and the trough in accordance 
^ with the expression (2) and the coordinate X* thus obtained is registered on the tabte (steps 214 and 3(4). 
Subsequently, this table is searched and the mean value U is obtained for the coordinate X* in accordance 
with the expression (3). 

When the central position taken in the direction of the measurement line was obtained for each array, 
the position of the entire IC and the contact pattern is obtained (step 324), Incidentally, in the flat package 

w IC. there are two types: in one of them, pins are led out in four directions and in the other, pins are led out 
in two directions. In correspondence thereto, there are two image patterns, as shown in Fig. 7 (a) and (b). In 
both case, two arrays opposed are parallel to each other. 

First, m the case of four directional pattern as shown in Fig. 7 (a), coordinates Ul, U2, VI and V2 are 
obtained respectively as the mean vaJue from four pattern arrays. In this case, UI and U2 and VI and V2 are 

w respectively connected by the straight line and their intersecting point (U, V) is defined as the cmfrai 
position of the whole pattern. Further, the direction ID of a bisector of two fines -is defined as the total 
direction. Thus this direction ID of the bisector offers the information on the offset of the rotation. 
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Next. In the case of the two directional pattern, as shown in Fig. 7 (b), only the accuracy taken in the 
direction of the measurement line is important and the accuracy taken in the direction intersecting at right 
angles therewith can be met enough by the mechanical positioning. Consequently, the whole position is 
obtained, since V Is fixed and U (Ul + U2)/2, and the direction of the whoie can be denned as the direction 
ID connecting Ul and U2. 

Thus positions of the flat package IC and the contact pattern on the board can be accurately obtained 
and in accordance with such positional information, the robot can accurately load the IC as 'the surfaced 
part onto the contact pattern. 

However, the pattern array is not necessarily regularly disposed and. there are also such cases in which 
some defects exist at a portion of the pattern array or the symmetry is 'mpaired for the pattern array on 
both sides viewed from the central position. These pattern arrays are not simply defective, but are often 
intentionally so formed. 

In such pattern arrays, even ff ideas of the recognition of the crest and the trough and the weighted 
mean is introduced, as described above, the central position of the pattern arrays cannot be accurately 
is detected. 

However, in the present invention, even if the pattern arrays are partially absent and asymmetrical, the 
central position can be accuratefy detected. The invention is hereinafter described in detail. 

A case where the width of pattern at both ends of the contact pattern array 16 on the board differs from 
other patterns, as shown in Fig. II is described. -The detected value of the edge on the external side of the 
pattern st both ends of the pattern array is neglected and the data table is prepared. Reference numeral 20 
shows a through hole and 2J a wiring pattern. In Fig. 12 (a), (b), as shown in Rg. 12 (c), number 0 is placed 
in the position of the data table in which XI and XIO are to be stored. In the Figs., reference numeral 16 
shows a contact pattern and 17 a measurement line. 

As shown in Rg. 13 (a), (b). if the first edge point of the pattern array is a trailing point, it is regarded as ' 
25 the inner edge of the first pattern, as shown in Rg. 13 (c) and XI is put onto the second data table to prepare 
the data table, as in the above-described way. 

There are cases where succeeding edge points having the gradient of the brightness with the same 
sign are detected, as shown in Rg. 14 (a), but in the case of the leading edges, the detecting pcint X2 within 
the last leading edge is regarded' as the edge of the pattern, and in the case of the trailing edge, the 
detecting point X5 within the first trailing edge is regarded as the edge of the pattern, and XJ and X6 are not 
regarded as the edge point. These are shown in Rg. 14 (b). Incidentally, in Rg. 14 (a), reference numeral 16 
shows a contact pattern and 17 a measuring line and 22 a resist portion. 

As described above, within the table having the odd number, values at the leading edge point are 
stored and values at the trailing edge points are stored within the table having the even number, so that the 
state of the pattern arrays may be accurately registered, as a result of which the center of the pattern array 
can be obtained as the mean value of the effective data which is registered into the table. 

Though the present embodiment relates to the case where the pattern array is brighter than the 
background. If the pattern is darker than the background, the above-described definition specified for storing 
data into the data table may be reversed. 

Furthermore, as shown in Rg. 15, there is a flat package IC in which leads arrays are asymmetrically 
disposed with respect to the central point of the lead anay to be obtained. In the case of the pattern array, 
as shown in Rg. 16 (a), from the above description, the waveform of the brightness on the measuring line 
and the edge data resuft as shown in Rg. 16 (a) and Rg. 16 (b). In that case, since the portion PA can be 
regarded as the omitted pattern array, edge data X3 and X4 at the portion of P8 are neglected and as 
shown in Rg. 16 (d), the table at that portion is set to zero and the data table is prepared anew (step 218). 
The central point of the array can be obtained by the mean value of the effective data of the data tabfe as 
described above (step 222). ' 

According to the present embodiment, when such modification of the omission is necessary to 
accelerate the processing at the time of the measurement, data on each measuring line as well as 'the 
modification value Ap shown in Rg. 17 are registered at the time of teaching (step 224) and at the time of 
the measurement, this correction value Ap is added to or subtracted from the provisional central point P' 
which can be calculated from the data tabfe of Rg. 16 (c). so that the central point P of the array may be 
calculated (steps 318, 320). At the time of teaching, to calculate Ap. the central point P and the pre-defined 
central point P' of the array are calcualted from the data table as in Rg. 16 (a), (b) (steps 220 and 222) 
& Though the portion at which the omission is modified for calculating the central point P of the anay (portion 
PB in Rg. f7) is manually set with a view to the flexibility of the system, if the pattern anay is previously 
determined, the information on the modification of the omission is registered within the image processor 
and the central point P of the array can be calculated automatically without teaching it manually. 
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After the centra! point P of each array (Se9 Fig. 22) has been obtained, as shown in fig. 18 (a), when 
leads are led out in four directions, the intersecting point obtained by respectively connecting PI and P3 and 
P2 and P4 is taken to be a central point of the entire region and the inclination SA formed by the 
connecting line P2-P4 and the X-axis of Ihe TV coordinate system (visual coordinate system) is taken to be 

5 a posture. Furthermore, when leads are led out in two directions, as shown in fig. 18 (b), the central point of 
the line connecting PI with P3 is taken to be a central point of the entire region and the inclination formed 
by this connecting line and the X-axis is taken to be a posture, asJs the above case, so that theT position 
and the posture can be measured with a high accuracy (step 324). In the figure, as described above, 
reference numeral 16 shows a contact pattern and 17 a measurement line. 

/o Furthermore, referring to the above-described threshold values D th of the gradient of the concentration 
which can be regarded as the edge; they show the same cycle in the direction of the measurement Hne with 
the same lead interval laterally as well as longitudinally from the characteristics of the lens, solid image pick 
up element of the TV camera and the circuit of the image input or the like, as in the leads array within the 
screen 24 shown in Fig. 19 (a), but if the TV scanning direction is as shown in the figure, the waveform of 

75 . the brightness of the measurement line 17 results as shown in Fig. 19 (c), its height AD (difference of the 
contrast) being different Thus in a single screen, threshold values D th of the gradient of the concentration 
are provided for each direction of the measurement line, so that the component of the noise at the 
background may be cut. thereby allowing only the normal edges to be extracted (steps 202 and 302) 

fig. 20 shows a flow chart of the processing at the time of teaching in progress within the image 
20 processor according to the present embodiment and fig. 21 a flow chart of the processing at the time of - 
measurement 

According to the present invention, positions not only of the IC having leads feet arrayed with an equal 
interval, but also of the IC flat package the number of the lead feet of which is asymmetrical with respect to 
the center can be detected and referring to the printed board, there is no need to apply the special mark to 

25 the package at the time of manufacturing and a high accuracy detection of the position can be realized, 
independently of the configuration of both ends of the contact pattern array. Furthermore, since, depending 
on the component of the frequency of the pattern array, the threshold values which can be regarded as the 
edgeof the pattern can be separately provided, the position can be stably detected, as a result of which the 
apparatus for automatically loading surfaced parts can be obtained having a high reliability and a 

a? adaptability. 

By the way, in such a case, if leads of the IC which are to be loaded onto the board are bent, they 
cannot be accurately connected with the wiring pattern on the board. To prevent this possibility, the bend of 
these leads is previously inspected, so that on(y the IC having the accurate configuration of leads may be 
placed onto the board. 

The bend of leads of this IC is now described with reference to Fig. 23. First Fig. 23 (a) is a plan view 
of the fiat package type IC and the reference numeral 31 shows a main body of the IC and 32 represents a 
lead. 32 a and 32 b are bent leads. Furthermore, Rg. 23 (b) is also a side eievationaJ view and 32 c and 32 
b are bent leads. The bend of the lead 32 shown in this Fig. 23 (a), i.e. the bend formed along the arrayed 
direction of the leads array is called the lateral bend and the bend of the lead shown in Fig. 23 (b). i.e. the 
bend in the direction forming a right angle relative to the arrayed direction of the leads array is called the 
longitudinal bend. 

A method of detecting the bend of leads of the IC according to the present invention is hereinafter 
described in detail with reference to the embodiment shown. 

Fig. 24 is an embodiment of the present invention and reference numeral 31 shows a flat package type 
IC, 33 an adsorbing hand, 34 a chuck for the gross positioning, 35 a television camera, 36 a lens, 37 an 
image pick up element and 38 an illuminating unit. 

This embodiment is applied to the apparatus for automatically loading the IC and the adsorbing hand 33 
ts provided with the chuch 34, and chcks and holds the IC 31 which is previously provided at a 
predetermined position and carries it for positioning to a predetermined location of the board which is also 
previously set at a predetermined position. In this case, the chuck serves to roughly decide the posture at 
which the IC 31 is chcked to the adsorbing hand 33. 

At this time, this adsorbing hand 33 stops temporarily at the predetermined position, as described 
above, on the way to carrying the IC 31 from the above-mentioned predetermined position onto the board. 

Meanwhile, the television camera 35 is installed immediately below the adsorbing hand 33 in correspon- 
dence to the position where this adsorpting hand 33 is once stopped, so that the image of this IC may be 
formed by the tens 36 precisely onto the image pick up element 37. The IC 31 which is in the adsorbed and 
held state is picked up at the timing when {his hand 33 has just stopped and the data on the image is taken 
in. so that ft may be stored In the memory (frame memory) (not shown). 
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By the processing of data on the image stored within this memory, the position and the posture when 
the IC 31 is adsorbed to the hand 33 is detected and subsequently the positioning and the correction are 
controlled on the board, while, according to this embodiment, the bend of leads of the IC 31 is detected and 
it is decided whether that IC is defective or not When decided to be defective, this IC is removed. 
6 Next how the bend of leads of the IC is detected in accordance with the processing of the data on the 
image is described. 

As described already in Fig. 23, there are a lateral bend (Fig. 23 (a)) and a .longitudinal bend (Fig. 23 - 
(b)) for the. bend of leads of the IC. 

First, how the lateral bend is detected is described. 
70 The image of the IC which was picked up as in Fig. 24 is as shown in Fig. 25 (a). Within this image, the 
measuring line is previously, provided at the predetermined position and the data extracted along this 
measuring line 17 is prepared. 

Then the brightness of this data on the image draws a pattern as shown in Fig, 25 (b). 
Therefore, out of the data on the image which are stored in the form of the lattice within the frame 
memory, the data D at the portion corresponding to this measuring line 17 are weightedly averaged In 
accordance with the expression (2) and that value is taken as the coordinate X* at the edoe portion of the 
lead. 

Next, the lead pitch Ip (=]Lp -L^, |) between adjacent leads is obtained from these coordinates at the 
edge to detect the existence of the bend of the lead and from that amount, -it is decided whether that IC is 
20 defective or not The decision at .that time is conducted in accordance with the following expression (4) and 
if the condition of this expression (4) is not met, the IC is decided to be defective. 
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Consequently, according to this embodiment, if. as shown above, the absolute value |Lp -L(p +])[ of the 
difference of the lead pitch between adjacent leads is decided, the amount of the bend of the lead can be 
obtained and the data on the image obtained by picking up the image at the time when the IC was loaded 
onto the board can be used as they are to easily detect the bend of the lead. 

Next, how the longitudinal bend is detected Is described. Returning to Fig. 24, in this embodiment the 
image of the IC 31 is picked up by illuminating it by means of a ring light source device 38. The light source 
38 uses the optical fiber, so that the fight may be illuminated from the small array of the light sources which 
can be regarded substantially as the point fight source which is disposed around the image pickup lens 36 
in such a way that it may continuously surround this lens. 

How the image of the lead 32 is formed by this optical system according to the embodiment is shown 
in Fig. 26. 

Reference numeral 40 shows an aperture for emitting the light by the optical fiber ring of the 
Illuminating unit 38. 41 an optical center of the lens, 42 an optical axis of the lens. 43 en image pick up 
plans of the image pickup element 38. 44 a front principal point and 45 also a rear principal point. 
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First, beams c and illuminated from the aperture 40 for emitting the light are regularly reflected at the 
rear surface of the lead 32 and that reflected light reaches the image pickup plane via the lens 36 and the 
image is formed there. 

Consequently, since the brightness of the image of the lead 32 on the image pickup plane at this time 
5 is changed, depending on the angle e from the horizontal plane of this lead, i.e. the amount of the bend of 
this lead 32, since the aperture 40 is sufficiently small and can be regarded as the point light source. 

At this time, the angle when the image of the lead at the image.pickup plane 43 displays the maximum 
amount of light is decided in accordance with the following formula 



(5) 



(6) 

where: a: distance from the optical axis of the lens up to the reflected plane (point) of the lead 
20 b: distance from the aperture up to the reflection plane (point) of the lead 

c: distance from the optical axis of the lens up to the aperture 

d: distance from the reflection plane (point) of the lead up to the optical center of the lens 
Consequently, according to this embodiment, if the construction of each portion is set, so that 
appropriate values are given for the constants a. b, c and d in the expressions (5) and (6), only the amount 
25 of light reflected by the IC the amount of the angte 8 of which exceeds the predetermined value and which 
is to be decided to be defective can be set to be below the predetermined value and the longitudinal bend 
can be detected. 

This detection is conducted by setting the threshold value 0^ of the gradient of the concentration in the 
expression (I) to a predetermined value. The number of the portion exhibiting the amount of the light above 
friis threshold value D !h is compared with the number of leads previously given and if they coincide with 
each other, that article can be decided to be excellent 

As evident from the above description, the above described decrease, of the amount of light occurs due 
also to the absence of leads. Consequently, according to this embodiment the IC in which leads are 
missing due to the damage or the like can also be simultaneously detected. 

Fig. 27 schematically shows a flow chart, illustrating how the bend of leads is detected in the above- 
described embodiment. 

First of the data on the image which are stored within the frame memory, the one disposed along the 
measurement line L of a certain group of leads are transferred to a predetermined buffer. 
_ In steps 412 and 4H, in accordance with the step 408, the coordinate of the edge is detected in 
accordance with expressions (I) and (2), and the results are sequentially tabulated. The process up to these 
steps 402 to 416 is identical with the one up to the steps 302 to 316. In the step 418, the pitch distance is 
calculated in accordance with the expression 4 and the result is decided in the following step 420 to detect 
the lateral bend. 

In steps 422 and 424, positioning the IC onto the board which is intrinsically asked of the automatic 
loading apparatus to which this embodiment is applied is conducted. 

In the step 426, it is decided whether the above^J escribed processes have been completed for the 
entire leads array of the IC. Namely, in the IC, there are generally a plurality of lead arrays, as is the case 
with the IC shown in Fig. 23 (four arrays). Therefore, processing is repeated for each measuring line in . 
correspondence to these plurality of lead array. Incidentally, In almost all cases, the number of the lead 
array is generally two and four. 

In the step 428, the variation of the amount of light which is obtained by the expressions (5) and (6) is 
decided by the threshold value of the expression (12) and the number of leads is checked for deciding 
whether the article is defective or not in accordance with that result. 

In the step 430, the position and the posture at which the IC is held by the adsorbing hand 33 (Fig. 24) 
are detected in accordance with the result obtained by the processing 424. 



Q a = 1/2 [2tt.- Tan cl/a 
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Consequently, according to this embodiment, since, at the time of loading the IC automatically onto the 
board, the bend of leads can be detected only by using the data on the image which are used for the 
control of this process, this invention can be applied to the automatical loading appanrtus and the so-called 
tact time (time required for series of processings needed for mounting one !C onto the board) is not almost 
increased and the bend of leads can be correctly detected. 

Furthermore, in the above-described embodiment though one measuring line 17 is set for each lead 
array of the IC, a plurality of the measuring lines may be instead provided to conduct plural measurements 
in accordance wrth them and of the result detected by these measurement lines, the data which have 
displayed the maximum number of leads may be used to carry out the finaJ detection. According to this 
» system, the reliability of the article can be further improved. 

Claims 

1 . A method of loading surface mounted device for loading said surface mounted device chucked by 
the hand of the automatic control apparatus onto the contact pattern on the board by said automatic 
apparatus, comprising steps of; 

providing the measurement line on each Image pattern of the array of the contact patterns, of the surfaced 
part, in such a way that it may intersect substantially at right angles with the .direction of the pattern, 
Obtaining each position of said array of the contract pattern from the variation of the brightness distributed 
on said measurement line. 

Obtaining the displacements of the position and the rotation of the surfaced parts and the entire contact 
pattern in accordance with said position, and, 

Controlling the operation of said automatic apparatus at the time of loading in accordance with the 
information on said displacements, 

2. A method of loading surface mounted devicve, as defined in Claim 1, wherein said step of obtaining 
each position of the array of the contact pattern comprises; 

specifying the position of the edge with respect to each boundary of said array of the contact pattern, 
calculating the coordinate of the center of the pin array or the contact pattern from said position of the 
edge, taking into consideration the asymmetry and the partial absence of said contact pattern. 

3. A method of loading surface mounted device, as defined in Claim 1, comprising a further step of 
detecting the bend of each pin of pin anay corresponding to said array of the contact pattern from the 
variation of the brightness distributed on said measurement line. 

4. A method of loading surface mounted device, as defined in Claim 3, wherein said surfaced part is a 
flat package IC and said method comprises a step of deciding the lateral bend of pins of said IC which 
occurs along the arrayed direction of the pin array and a step of deciding the longitudinal bend which 
emerges in the direction intersecting at right angles with the arrayed direction of the pin array. 

5. A method of loading surfaced mounted device, as defined in Claim 1, wherein the object is 
illuminated by a plurality of the point light sources continuously disposed around the image pickup lens in 
such a way that said sources may surround said lens at the time of image pickup for obtaining data on the 
plane image of said surfaced parts. 

6. A method of loading surface mounted device, as defined in Claim 5, wherein the light emitted from 
the end surface of the optical fiber is used as said point light source. 

7. A loading apparatus for loading the IC onto the printed board by registering a plurality of the contact 
patterns on the printed board wrth a plurality of leads on the IC, comprising; 

an image pickup means (1,2) for picking up the image of said contact pattern or the leads, 
a memory means for setting the measurement line onto said picked up contact pattern or leads, and storing 
the data on the brightness distributed along said measurement line in the order in which patterns or leads 
are arrayed, 

a processing means (7) for calculating the coordinate of the center (P) of the contact pattern or lead from 
the data on the brightness (UU) stored within said memory means, and, 

a means for specifying the position and the inclination of the printed board or the IC from the coordinate of 
the center (P) obtained by said processing means to register said printed circuit with said IC. 

8. A loading apparatus, as defined in Claim 7, wherein said processing means specifies each edge of 
the contact pattern or leads from said data on the brightness (I.IJ) and calculates the coordinate of the 
center (P) of the contact pattern or the lead from said coordinate of the edge, taking 'into consideration the 
asymmetry or the partial absence of said contact pattern or the lead. 
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9. A loading apparatus, as defined in Claim 8, wherein, in the presence of said asymmetry and the 
partial absence, the modification data are previously stored within the memory means, so that the 
coordinate of the center (P) may be calculated from said modification data and the provisional center of the 
coordinate of said edge. 

10. A loading apparatus, as defined In Claim 7 t wherein a detecting means is provided for detecting the 
bend of each of said leads in accordance with the data stored within said memory means. 

11. A loading apparatus, as defined in Claim 10, wherein said detecting means comprises a means for 
detecting the lateral bend which emerges along the arrayed direction of the lead array of the IC (31) and a 
means for detecting the longitudinal bend which emerges in the direction intersecting at right angles with 
the an-ayed direction of the lead array. 

12. A loading apparatus, as defined in Claim 10, ydyemn an adsorbing hand (33) is provided for shifting 
the IC (31) to a predetermined mounting position and positioning it and the bend of said (C (31) is detected 
in accordance with the data on the picked up plane image of the IC (31) which is positioned and held by 
said hand.(33) 

13. A loading apparatus, as defined in Claim 7, wherein the object is Illuminated by a plurality of the 
point light sources (38) continuously disposed around the image pickup lens (36) in such a way that they 
may surround said lens (38) at the time of image pickup for obtaining the data on the image of the plane of 
said IC.(3I) 

14. A loading apparatus, as defined in Claim 13, wherein the light emitted from the end surface of the 
optical fiber is used as said point light source.(38) 

15. A loading method for loading the IC onto the board comprising the steps of; 

(a) when the board is mechanically fixed on the board supply apparatus, the TV camera which is 
fixed to the hand of the robot being moved onto said board and seeking the position of the contact pattern 
for loading the IC on the board from the variation of the brightness distributed on the measurement line 
which is provided in such a way thai it may intersect at right angles with the direction of the pattern, 

(b) the hand being moved to the unit for positioning the gross position of the IC and adsorbing the IC 
(integrated circuit) from said apparatus, 

(c) the IC being moved onto the TV camera for detecting the position of the IC, so that the 
displacement between the IC and the hand is detected, 

(d) positions of the contact pattern and the IC which are obtained by the image processor being fed 
back via the robot control device to the robot, so that the IC may be loaded onto the contact pattern on the 
board in the excellently registered way. 



0 222 072 



FIG. 1 
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FIG. 4 
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